Spark is a distributed data processing framework based on memory. Memory allocation is a focus question of Spark research. A good memory allocation scheme can effectively improve the efficiency of task execution and memory resource utilization of the Spark. Aiming at the memory allocation problem in the Spark2.x version, this paper optimizes the memory allocation strategy by analyzing the Spark memory model, the existing cache replacement algorithms and the memory allocation methods, which is on the basis of minimizing the storage area and allocating the execution area according to the demand. It mainly including two parts: cache replacement optimization and memory allocation optimization. Firstly, in the storage area, the cache replacement algorithm is optimized according to the characteristics of RDD Partition, which is combined with PCA dimension. In this section, the four features of RDD Partition are selected. When the RDD cache is replaced, only two most important features are selected by PCA dimension reduction method each time, thereby ensuring the generalization of the cache replacement strategy. Secondly, the memory allocation strategy of the execution area is optimized according to the memory requirement of Task and the memory space of storage area. In this paper, a series of experiments in Spark on Yarn mode are carried out to verify the effectiveness of the optimization algorithm and improve the cluster performance. 相似文献
Differential evolution is primarily designed and used to solve continuous optimization problems. Therefore, it has not been widely considered as applicable for real-world problems that are characterized by permutation-based combinatorial domains. Many algorithms for solving discrete problems using differential evolution have been proposed, some of which have achieved promising results. However, to enhance their performance, they require improvements in many aspects, such as their convergence speeds, computational times and capabilities to solve large discrete problems. In this paper, we present a new mapping method that may be used with differential evolution to solve combinatorial optimization problems. This paper focuses specifically on the mapping component and its effect on the performance of differential evolution. Our method maps continuous variables to discrete ones, while at the same time, it directs the discrete solutions produced towards optimality, by using the best solution in each generation as a guide. To judge its performance, its solutions for instances of well-known discrete problems, namely: 0/1 knapsack, traveling salesman and traveling thief problems, are compared with those obtained by 8 other state-of-the-art mapping techniques. To do this, all mapping techniques are used with the same differential evolution settings. The results demonstrated that our technique significantly outperforms the other mapping methods in terms of the average error from the best-known solution for the traveling salesman problems, and achieves promising results for both the 0/1 knapsack and the traveling thief problems. 相似文献
A novel couple-based particle swarm optimization (CPSO) is presented in this paper, and applied to solve the short-term hydrothermal scheduling (STHS) problem. In CPSO, three improvements are proposed compared to the canonical particle swarm optimization, aimed at overcoming the premature convergence problem. Dynamic particle couples, a unique sub-group structure in maintaining population diversity, is adopted as the population topology, in which every two particles compose a particle couple randomly in each iteration. Based on this topology, an intersectional learning strategy using the partner learning information of last iteration is employed in every particle couple, which can automatically reveal useful history information and reduce the overly rapid evolution speed. Meanwhile, the coefficients of each particle in a particle couple are set as distinct so that the particle movement patterns can be described and controlled more precisely. In order to demonstrate the effectiveness of our proposed CPSO, the algorithm is firstly tested with four multimodal benchmark functions, and then applied to solve an engineering multimodal problem known as STHS, in which two typical test systems with four different cases are tested, and the results are compared with those of other evolutionary methods published in the literature. 相似文献
近似计算技术通过降低电路输出精度实现电路功耗、面积、速度等方面的优化.本文针对RM(Reed-Muller)逻辑中"异或"运算特点,提出了基于近似计算技术的适合FPRM逻辑的电路面积优化算法,包括基于不相交运算的RM逻辑错误率计算方法,及在错误率约束下,有利于面积优化的近似FPRM函数搜索方法等.优化算法用MCNC(Microelectronics Center of North Carolina)电路进行测试.实验结果表明,提出的算法可以处理输入变量个数为199个的大电路,在平均错误率为5.7%下,平均电路面积减少62.0%,并在实现面积优化的同时有利于实现电路的动态功耗的优化且对电路时延影响不大. 相似文献
Objective: Paclitaxel (PTX)-loaded polymer (Poly(lactic-co-glycolic acid), PLGA)-based nanoformulation was developed with the objective of formulating cremophor EL-free nanoformulation intended for intravenous use.
Significance: The polymeric PTX nanoparticles free from the cremophor EL will help in eliminating the shortcomings of the existing delivery system as cremophor EL causes serious allergic reactions to the subjects after intravenous use.
Methods and results: Paclitaxel-loaded nanoparticles were formulated by nanoprecipitation method. The diminutive nanoparticles (143.2?nm) with uniform size throughout (polydispersity index, 0.115) and high entrapment efficiency (95.34%) were obtained by employing the Box–Behnken design for the optimization of the formulation with the aid of desirability approach-based numerical optimization technique. Optimized levels for each factor viz. polymer concentration (X1), amount of organic solvent (X2), and surfactant concentration (X3) were 0.23%, 5?ml %, and 1.13%, respectively. The results of the hemocompatibility studies confirmed the safety of PLGA-based nanoparticles for intravenous administration. Pharmacokinetic evaluations confirmed the longer retention of PTX in systemic circulation.
Conclusion: In a nutshell, the developed polymeric nanoparticle formulation of PTX precludes the inadequacy of existing PTX formulation and can be considered as superior alternative carrier system of the same. 相似文献
A novel prediction and optimization method based on improved generalized regression neural network (GRNN) and particle swarm optimization (PSO) algorithm is proposed to optimize the process conditions for styrene epoxidation to achieve higher yields. This model was designed to optimize the five input parameters reaction temperature and time as well as catalyst, solvent, and oxidant dosage. The output of the improved GRNN was given to the PSO algorithm to optimize the process conditions. The optimal smoothing parameter σ of GRNN was chosen from the training sample with a minimum cross validation error. Under the five optimized process conditions the maximum yield reached 95.76 %. This innovative model of improved GRNN hybrid PSO algorithm proved to be a useful tool for optimization of process conditions for styrene epoxidation. 相似文献